Reducing false alarms in surveillance systems

ABSTRACT

A surveillance system includes an infrared sensor system coupled to output an infrared signal in response to receiving infrared light, and an audio recording system coupled to output an audio signal in response to recording sound. An image sensor is system coupled to output an image signal in response to receiving image light. A controller is coupled to the infrared sensor system, the audio recording system, and the image sensor system. The controller includes logic that when executed by the controller causes the surveillance system to perform operations including receiving the infrared signal from the infrared sensor system, activating the audio recording system to record the sound, and activating the image sensor system to output the image signal.

TECHNICAL FIELD

This disclosure relates generally to surveillance systems.

BACKGROUND INFORMATION

Cameras for the home and business are becoming popular consumerproducts. Cameras may be a cost-effective way to have a surveillancesystem at a relatively low price.

Existing camera systems may suffer from undesirable technologicalshortcomings. Some camera systems may turn on to record insignificantevents based on false positives or the like. For example, motionactivated cameras may turn on when there is no person or thing moving.This may cause the camera system to waste power.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive examples of the invention are describedwith reference to the following figures, wherein like reference numeralsrefer to like parts throughout the various views unless otherwisespecified.

FIG. 1 illustrates a surveillance system, in accordance with an exampleof the present invention.

FIG. 2A illustrates an infrared sensor system included in thesurveillance system of FIG. 1, in accordance with an example of thepresent invention.

FIG. 2B illustrates an audio recording system included in thesurveillance system of FIG. 1, in accordance with an example of thepresent invention.

FIG. 2C illustrates an image sensor system included in the surveillancesystem of FIG. 1, in accordance with an example of the presentinvention.

FIGS. 3A-3D illustrate methods of operating the surveillance system ofFIG. 1, in accordance with examples of the present invention.

FIG. 4 illustrates a method of operating the surveillance system of FIG.1, in accordance with an example of the present invention.

Corresponding reference characters indicate corresponding componentsthroughout the several views of the drawings. Skilled artisans willappreciate that elements in the figures are illustrated for simplicityand clarity and have not necessarily been drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help to improve understanding of variousembodiments of the present invention. Also, common but well-understoodelements that are useful or necessary in a commercially feasibleembodiment are often not depicted in order to facilitate a lessobstructed view of these various embodiments of the present invention.

DETAILED DESCRIPTION

Examples of a system and method for reducing false alarms insurveillance systems are described herein. In the following description,numerous specific details are set forth to provide a thoroughunderstanding of the examples. One skilled in the relevant art willrecognize, however, that the techniques described herein can bepracticed without one or more of the specific details, or with othermethods, components, materials, etc. In other instances, well-knownstructures, materials, or operations are not shown or described indetail to avoid obscuring certain aspects.

Reference throughout this specification to “one example” or “oneembodiment” means that a particular feature, structure, orcharacteristic described in connection with the example is included inat least one example of the present invention. Thus, the appearances ofthe phrases “in one example” or “in one embodiment” in various placesthroughout this specification are not necessarily all referring to thesame example. Furthermore, the particular features, structures, orcharacteristics may be combined in any suitable manner in one or moreexamples.

The disclosure herein relates to reducing power consumption ofsurveillance systems (see infra surveillance system 100 of FIG. 1) bydecreasing the system's controller usage rate. To reduce powerconsumption, an acoustic activity detector (see infra audio recordingsystem 103 of FIG. 1) may be implemented in a digital integrated circuitcoupled with audio codec functionalities. This may reduce the number offalse alarms that the surveillance system registers. The acousticactivity detector is activated after a passive infrared (IR) motionsensor senses IR light. The acoustic activity detector may then activatesophisticated computer vision algorithms running on the controller oncethe system is awake.

This hierarchical activation of the surveillance system may prevent thefull system from turning on in the event of a false alarm. For example,if just an IR sensor was used, excessive sunlight may cause the securitysystem to turn on and drain the batteries. Accordingly, each of thesurveillance systems disclosed herein my use progressively more power toverify that a security event is in fact real (e.g., a person, vehicle,or animal activating the system).

Since moving objects generate both motion and sound activity, humans areable to determine if someone is walking near them by analyzing bothvisual and auditory information. The devices herein use machine analogsof these biological systems to prevent false positives. The electronicshere realize acoustic activity detection (AAD) circuitry, and the AADcan use a digital circuit to extend battery life of portable deviceslike a doorbell camera. Using motion and acoustic sensors allows thecontroller (e.g., the brain of the system) to only be triggered in caseswhere the infrared sensor circuit (e.g., the eyes of the system) detectsmotion, and the acoustic detection circuit (e.g., the ears of thesystem) detects activities.

In some examples, a comprehensive detection system to monitor visual andaudio signals is proposed that avoids false alarms while saving power. Afirst level detection system (always awake) employs ultra-low powerpassive IR sensing (which may be prone to false alarms). A second leveldetection system (which may be awakened by the first level detectionsystem) uses low-power noise based acoustic activity detection (e.g.,employing infinite impulse response filter circuitry) to “check” thefirst level system and prevent false alarms. The third level detectionsystem employs normal power advanced sound monitoring, which may run ona BA-22 processor based digital signal processor (DSP). And the fourthlevel detection system may include high power computer vision algorithmsrunning on a high-performance processor (e.g., with central processingunit and/or visual processing unit). Multiple detection levels help toavoid false alarms, and the levels are arranged in tiers, with moreadvanced (but power-consuming) detection positioned further downstream.This may be used to save power in battery operated doorbell securitysystems or the like.

The following description discusses the examples discussed above, aswell as other examples, as they relate to the figures.

FIG. 1 illustrates a surveillance system 100, which includes infraredsensor 101, audio recording system 103, button 105, image sensor system107, housing 109, controller 111, power circuitry 113, communicationcircuitry 115, and memory 117. In the illustrated example, all of thecomponents depicted (and components that are not depicted here) may beelectrically coupled together via a bus or the like.

As shown, infrared (IR) sensor system 101 is coupled to output aninfrared signal in response to receiving infrared light, and audiorecording system 103 is coupled to output an audio signal in response torecording sound. Image sensor system 107 is coupled to output an imagesignal in response to receiving image light (e.g., light from an imagesubject, such as a person, animal, or vehicle). Controller 111 (whichmay include a central processing unit (CPU), a vision processing unit(VPU), and may be part of a distributed system, or the like) is coupledto infrared sensor system 101, audio recording system 103, button 105,and image sensor system 107. Controller 111 includes logic that whenexecuted by controller 111 causes surveillance system 100 to perform avariety of operations. For example, operations may include receiving theinfrared signal from infrared sensor system 101, activating audiorecording system 103 (in response to receiving the infrared signal) torecord sound, and activating image sensor system 107 to receive imagelight and output the image signal in response to recording a specificsound profile. In some examples, controller 111 may be coupled tocommunication circuitry 113 (e.g., WiFi, RFID, 8P8C compatible circuitryor the like) to output a notification to one or more external devices(e.g., one or more computers/servers on a network, a cell phone, tablet,or the like) after audio recording system 103 records the sound. In someexamples, the notification may be output after the image signal isoutput to the controller from image sensor system 107. The notificationmay include at least one of a text message, a video feed, a phone call,a sound notification, or the like.

In the depicted example, memory 117 (e.g., RAM, ROM or the like) may beincluded in (or coupled to) controller 111, and memory 117 is coupled tostore the infrared signal, the audio signal, and the image signal.Similarly, power circuitry 113 (e.g., battery, switched mode powersupply, power converter, or the like) is coupled to supply power toinfrared sensor system 101, audio recording system 103, image sensorsystem 107, and controller 111. As shown, infrared sensor system 101,audio recording system 103, image sensor system 107, controller 111,memory 117 and power circuitry 113 are disposed in single housing 109.Housing 109 depicted here may be positioned on the entrance of a home orbusiness to observe and control ingress and egress to/from the building.Housing may be coupled to the building power supply, a battery, or thelike. Button 105 may be used as a doorbell to inform people inside thebuilding (or remote individuals via a smartphone or the like) thatsomeone is at the door. Pressing button 105 may cause a notification tobe sent from surveillance system 100. Controller 111 may receive a presssignal, and the press signal causes the controller to perform operationssuch as activating any one of the components described above or sendinga notification.

In some examples, controller 111 may recognize at least one of a personor an animal in the image signal from image sensor system 107 using acomputer vision algorithm disposed in the logic in controller 111. Thismay prompt a notification to an external device (e.g., a home owner'ssmart phone or the like) from surveillance system 100 that a person oranimal is near the house. In some examples, the notification is outputto a wireless Internet network using a wireless router included incommunication circuitry 115.

FIG. 2A illustrates an infrared sensor system 201 included in thesurveillance system of FIG. 1. As illustrated, infrared sensor 225 iscoupled to output the infrared signal in response to receiving theinfrared light. As shown, Fresnel lens 221 is positioned to directinfrared light into infrared sensor 225 through IR filter 223. Amplifier227 is coupled to infrared sensor 225 to amplify the infrared signal,and comparator 229 is coupled to compare the infrared signal (after theinfrared signal is amplified) to a threshold value. When the infraredsignal is greater than or equal to the threshold value, comparator 229outputs the infrared signal to the controller. In the depicted example,a first resistor is coupled between input and output terminals ofinfrared sensor 225. A gate terminal of a transistor is coupled to aterminal of the infrared sensor 225 and a first terminal of thetransistor is coupled to a power rail and a second terminal of thetransistor is coupled to amplifier 227 and a second resistor.

FIG. 2B illustrates an audio recording system 203 included in thesurveillance system of FIG. 1. In the depicted example, microphone 241is coupled to output an audio signal in response to recording a sound.In the depicted example, mic 241 is coupled to a programmable gate array243 to amplify the output of mic 241. Programmable gate array 243 iscoupled to analog to digital converter 245 to convert the amplifiedanalog signal into a digital signal. The microphone 241 is coupled to anacoustic activity detection (AAD) circuit (that is signal to noise (SNR)based) to receive the audio signal. The output of ADC 245 is sent tosignal-tracking filter circuit 251, and noise-tracking filter circuit253. The outputs of filter circuit 251 and filter circuit 253 arecoupled to signal-to-noise calculator 255, which is coupled to receive aprogrammable threshold signal-to-noise ratio (SNR) value 257. In thedepicted example, the threshold is 10 dB.

FIG. 2C illustrates an imaging system 207 (or more specifically an imagesensor system) included in the surveillance system of FIG. 1. Imagingsystem 207 includes pixel array 205, control circuitry 221, readoutcircuitry 211, and function logic 215. In one example, pixel array 205is a two-dimensional (2D) array of photodiodes, or image sensor pixels(e.g., pixels P1, P2 . . . , Pn). As illustrated, photodiodes arearranged into rows (e.g., rows R1 to Ry) and columns (e.g., column C1 toCx) to acquire image data of a person, place, object, etc., which canthen be used to render a 2D image of the person, place, object, etc.However, the rows and columns do not necessarily have to be linear andmay take other shapes depending on use case.

In one example, after each image sensor photodiode/pixel in pixel array205 has acquired its image data or image charge, the image data isreadout by readout circuitry 211 and then transferred to function logic215. Readout circuitry 211 may be coupled to readout image data from theplurality of photodiodes in pixel array 205. In various examples,readout circuitry 211 may include amplification circuitry,analog-to-digital (ADC) conversion circuitry, or otherwise. Functionlogic 215 may simply store the image data or even alter/manipulate theimage data by applying post image effects (e.g., crop, rotate, removered eye, adjust brightness, adjust contrast, or otherwise). In oneexample, readout circuitry 211 may readout a row of image data at a timealong readout column lines (illustrated) or may readout the image datausing a variety of other techniques (not illustrated), such as a serialreadout or a full parallel readout of all pixels simultaneously.

In one example, control circuitry 221 is coupled to pixel array 205 tocontrol operation of the plurality of photodiodes in pixel array 205.Control circuitry 221 may be configured to control operation of thepixel array 205. For example, control circuitry 221 may generate ashutter signal for controlling image acquisition. In one example, theshutter signal is a global shutter signal for simultaneously enablingall pixels within pixel array 205 to simultaneously capture theirrespective image data during a single acquisition window. In anotherexample, the shutter signal is a rolling shutter signal such that eachrow, column, or group of pixels is sequentially enabled duringconsecutive acquisition windows. In another example, image acquisitionis synchronized with lighting effects such as a flash.

FIGS. 3A-3C illustrate methods of operating the surveillance system ofFIG. 1. As described above, any of these techniques may be used tooperate the hardware of the surveillance system of FIG. 1, in accordancewith the teachings of the present disclosure. One of ordinary skill inthe art will appreciate that the blocks depicted in methods 300A-300Dmay occur in any order, and even in parallel, and that steps andfeatures may have been omitted for simplicity of illustration. Moreover,additional blocks may be added to, or removed from, methods 300A-300D inaccordance with the teachings of the present disclosure.

FIG. 3A depicts a first method 300A for operating the surveillancesystem. Block 301 show first measuring passive IR light with an infraredsensor system. In this example, the IR sensor system may output aninfrared signal to the system controller. In the depicted example, theIR sensor system may consume 0.35 mA of power.

Block 303 illustrates, in response to receiving the infrared signal(e.g., when the IR sensor receives a threshold amount of IRradiation—e.g., a luminance threshold, a duration threshold, or thelike), activating an audio recording system, where the audio recordingsystem is coupled to output an audio signal to the controller inresponse to recording a sound. In the depicted example, the audiorecording system may consume 01-1.5 mA of power.

Block 305 illustrates activating an image sensor system to receive imagelight, where the image sensor system is coupled to the controller tooutput an image signal. In the depicted example, the image sensor systemis activated in response to the audio recording system recording a soundwith particular threshold characteristics (e.g., volume, duration,waveform, or the like), and outputting an audio signal to thecontroller. In the depicted example, the image sensor system may consume100-300 mA of power.

FIG. 3B depicts a second method 300B for operating the surveillancesystem. Like FIG. 3A, block 301 illustrates measuring passive IR lightwith an infrared sensor system. This may be achieved by receivinginfrared light, focused by a Fresnel lens and passed through an infraredlight filter, with an infrared sensor coupled to output an infraredsignal.

Block 303 depicts activating an active audio detection system. This maybe used to determine if the IR light disturbance (that caused the systemto start the activation sequence) is an event that is worth waking othercomponents and using additional energy to record, or if the event isjust a false alarm (e.g., the IR sensor getting too much sunlight).

Block 305 depicts then activating advanced sound monitoring if the lowpower active audio detection system registers sound with a thresholdcharacteristic (e.g., volume, duration, or the like). The advanced soundmonitoring circuitry consumes more power than the active sound detectioncircuitry, and may have additional functionality (e.g., examines thewaveforms of the sounds to see if the sound is characteristic of anevent such as footsteps, talking, or the like). Advanced soundmonitoring may employ machine-learning algorithms, such as a neural netor the like, to characterize a sound. Activating the audio recordingsystem may include activating the active sound detection circuitry, andthen the advanced sound monitoring circuitry at the same or a differenttime.

Block 307 shows activating an image sensor system to receive image lightand output image data. In some examples, in response to receiving theimage signal with the controller, the controller may recognize at leastone of a person or an animal in the image signal using a computer visionalgorithm disposed in the logic. In some examples, machine-learningalgorithms, such as a convolutional neural net or the like, may be usedto analyze image data and determine the contents of the images. It isappreciated that while the controller here is described as a discretedevice, the controller may be a distributed system, where someprocessing occurs locally (e.g., across many pieces of hardware), and/orsome processing occurs remotely (e.g., on many remote severs), or thelike.

FIG. 3C depicts a third method 300C for operating the surveillancesystem. In the depicted example, block 301 shows measuring passive IRlight. Unlike the other examples, block 303 shows commencing advancedsound monitoring (e.g., higher power calculations) immediately afterdetermining an event occurred using the passive IR sensor. Like theother examples, block 305 shows activating computer vision afterverifying the security disturbance via processing of the sound.

FIG. 3D depicts a fourth method 300D for operating the surveillancesystem. In the depicted example, block 301 shows measuring passive IRlight. However, blocks 303 and 305 show that the image sensor system isactivated at the same time as the audio recording system. Thus, in someexamples, these systems may be used in parallel to determine that asecurity disturbance is “real”.

FIG. 4 illustrates a (more detailed) method 400 of operating thesurveillance system of FIG. 1. One of ordinary skill in the art willappreciate that the blocks depicted in method 400 may occur in anyorder, and even in parallel, and that steps and features may have beenomitted for simplicity of illustration. Moreover, additional blocks maybe added to or removed from method 400 in accordance with the teachingsof the present disclosure.

Block 401 shows the system asleep. In this state the system may be in avery low power mode, with only the passive IR (PIR) sensor system turnedon.

Block 403 shows passively calculating the quantity of IR light receivedby the IR sensor system. This may occur when the system is “asleep”.

Block 405 shows determining if the IR light received is greater than athreshold amount of IR light. If the amount of IR light received is lessthan the threshold, the system remains in sleep mode.

Block 407 depicts if the IR light is greater than the threshold value,the system then turns on the audio recording system and begins activeaudio detection (AAD) by recording and listening for sounds.

Block 409 depicts the system calculating if the recorded sound surpassesa certain threshold (e.g., a volume threshold, a duration threshold, orthe like). If the answer is no, then the system may return to the sleepstate to conserve power.

If the recorded sound surpasses a certain threshold, block 411illustrates waking up the system.

Block 413 illustrates using the image sensor (and computer visiondisposed in logic) to determine if the image data contains somethreshold condition (e.g., the presence of a person or animal). If theimage does not contain the threshold condition, the system may revertback to the sleep state.

In block 415, if the image data does contain the threshold condition,the system may use the CPU/VPU running computer vision to generate aconfidence score: if the confidence score is above a certain thresholdthen the system may remain awake; if the confidence score is below acertain threshold then the system may reenter sleep mode.

The system may repeat method 400 many times. Moreover, the variousthreshold conditions may be checked at predetermined frequencies ordynamic frequencies (e.g., based on user input, power level, or thelike).

The above description of illustrated examples of the invention,including what is described in the Abstract, is not intended to beexhaustive or to limit the invention to the precise forms disclosed.While specific examples of the invention are described herein forillustrative purposes, various modifications are possible within thescope of the invention, as those skilled in the relevant art willrecognize.

These modifications can be made to the invention in light of the abovedetailed description. The terms used in the following claims should notbe construed to limit the invention to the specific examples disclosedin the specification. Rather, the scope of the invention is to bedetermined entirely by the following claims, which are to be construedin accordance with established doctrines of claim interpretation.

What is claimed is:
 1. A surveillance system, comprising: an infraredsensor system coupled to output an infrared signal in response toreceiving infrared light; an audio recording system coupled to output anaudio signal in response to recording a sound, wherein the audiorecording system includes active sound detection circuitry and advancedsound monitoring circuitry, wherein the advanced sound monitoringcircuitry consumes more power than the active sound detection circuitry;an image sensor system coupled to output an image signal in response toreceiving image light; and a controller coupled to the infrared sensorsystem, the audio recording system, and the image sensor system, whereinthe controller includes logic that when executed by the controllercauses the surveillance system to perform operations, including:receiving the infrared signal from the infrared sensor system; inresponse to receiving the infrared signal, activating the active sounddetection circuitry; in response to the active sound detection circuitryregistering the sound with a threshold characteristic, activating theadvanced sound monitoring circuitry to record the sound and to outputthe audio signal; and in response to receiving the audio signal outputby the advanced sound monitoring circuitry, activating the image sensorsystem to output the image signal.
 2. The surveillance system of claim1, further comprising communication circuitry coupled to the controllerto communicate with one or more external devices, and wherein thecontroller further includes logic that when executed by the controllercauses the surveillance system to perform operations, including:outputting a notification to the one or more external devices with thecommunication circuitry after the audio recording system records thesound.
 3. The surveillance system of claim 1, wherein the controllerfurther includes logic that when executed by the controller causes thesurveillance system to perform operations, including: in response toreceiving the image signal with the controller, recognizing at least oneof a person or an animal in the image signal using a computer visionalgorithm disposed in the logic.
 4. The surveillance system of claim 1,wherein the infrared sensor system includes: an infrared sensor coupledto output the infrared signal in response to receiving the infraredlight; a Fresnel lens positioned to direct the infrared light into theinfrared sensor; an amplifier coupled to the infrared sensor to amplifythe infrared signal; and a comparator coupled to compare the infraredsignal, after the infrared signal is amplified, to a threshold value,and when the infrared signal is greater than or equal to the thresholdvalue the comparator outputs the infrared signal to the controller. 5.The surveillance system of claim 1, wherein the threshold characteristicis a volume, a duration, or a waveform.
 6. The surveillance system ofclaim 1, wherein the image sensor system includes: an image sensorincluding a plurality of photodiodes; control circuitry coupled to theplurality of photodiodes to control operation of the plurality ofphotodiodes; and readout circuitry coupled to read out the image signal.7. The surveillance system of claim 1, further comprising: memorycoupled to the controller and coupled to store the infrared signal, theaudio signal, and the image signal; and power circuitry coupled tosupply power to the infrared sensor system, the audio recording system,and the image sensor system.
 8. The surveillance system of claim 7wherein the infrared sensor system, the audio recording system, theimage sensor system, the controller, the memory, and the power circuitryare disposed in a single housing.
 9. A method, comprising: receiving,with a controller, an infrared signal from an infrared sensor system,wherein the infrared sensor system is coupled to output the infraredsignal in response to receiving infrared light; in response to receivingthe infrared signal, activating active sound detection circuitry of anaudio recording system coupled to the controller; in response to theactive sound detection circuitry registering a sound with a thresholdcharacteristic, activating advanced sound monitoring circuitry of theaudio recording system to record the sound and to output an audiosignal, wherein the advanced sound monitoring circuitry consumes morepower than the active sound detection circuitry; and in response toreceiving the audio signal output by the advanced sound monitoringcircuitry, activating an image sensor system to output an image signal,wherein the image sensor system is coupled to the controller to outputthe image signal in response to receiving image light.
 10. The method ofclaim 9, further comprising outputting a notification to one or moreexternal devices with communication circuitry, coupled to thecontroller, after the audio recording system records the sound.
 11. Themethod of claim 10, wherein the notification is output to a wirelessinternet network using a wireless router included in the communicationcircuitry.
 12. The method of claim 9, wherein the thresholdcharacteristic is a volume, a duration, or a waveform.
 13. The method ofclaim 9, further comprising, in response to receiving the image signalwith the controller, recognizing at least one of a person or an animalin the image signal using a computer vision algorithm disposed in logicin the controller.
 14. The method of claim 9, wherein receiving infraredlight includes receiving infrared light passed through a Fresnel lensand an infrared light filter with an infrared sensor coupled to outputthe infrared signal.
 15. The method of claim 9, wherein recording thesound with the audio recording system includes using one or moremicrophones and an acoustic activity detection circuit coupled toreceive the audio signal.
 16. The method of claim 9, further comprising:receiving a press signal from a button with the controller in responseto the button being pressed, and activating the image sensor system inresponse the button being pressed.
 17. The surveillance system of claim1, wherein the advanced sound monitoring circuitry is configured toutilize a machine learning algorithm to characterize the sound.
 18. Thesurveillance system of claim 1, wherein the active sound detectioncircuitry comprises a signal to noise ratio-based acoustic activitydetection circuit.
 19. The method of claim 9, wherein activating theadvanced sound monitoring circuitry comprises utilizing a machinelearning algorithm to characterize the sound.
 20. The method of claim 9,wherein activating the active sound detection circuitry comprisesutilizing a signal to noise ratio-based acoustic activity detectioncircuit.